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« ARCHITECTURE FOR THE TRANSPORT OF MPEG-4 DATA OVER MPEG-2 SYSTEM" 

i 

FIELD OF THE INVENTION 

5 The present invention relates to a global end-to-end system architecture for 

the transport of MPEG-4 data over MPEG-2 Systems, It also relates, in such an 
architecture, to a server platform receiving an MPEG-2 transport stream and MPEG-4 data 
to be incorporated within said stream and to encapsulating devices for carrying out the 
encapsulation of said MPEG-4 data in this existing MPEG-2 transport stream. 

10 BACKGROUND OF THE INVENTION 

The document ISO/IEC-JTC1/SC29/WG11/N3050 ("Information technology - 
Generic coding of moving pictures and audio : systems", Amendment 7 : Transport of 
ISO/IEC- 14496 data over ISO/IEC- 138 18-1, January 2000) is considered as the 
technological background of the present invention. This document is enclosed (pp.2-31). 
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DESCRIPTION OF THE INVENTION 

The present invention is described in pp.32-45 and claimed in p.46. 



ISO/EEC 13818-l:199fi/Amd.7:19«>xffe) - 

4 INTERNATIONAL ORGANISATION FOR STANDARDISATION 

ORGANISATION INTERNATIONALE DE NORMALISATION 

ISO/DEC JTC1/SC29/WG1 1 
CODING OF MOVING PICTURES AND AUDIO 

ISO/EEC JTCl/SC29/WGllN3050 

_ ^ January 2000 

Systems 

Title: ISO/EEC 13818-1/FDAM 7 



INFORMATION TECHNOLOGY - 

GENERIC CODING OF MOVING PICTURES AND AUDIO: 
SYSTEMS 

Amendment 7: Transport of ISO/TEC 14496 data over 
ISO/TEC 13818-1 



ISO/TEC 13818-1/Final Draft Amendment 7 



International Standard 



ISO/EEC 13818-l:1996/Amd.7:199x(E) 



INTERNATIONAL STANDARD 
ITU-T RECOMMENDATION 

INFORMATION TECHNOLOGY - GENERIC CODING OF MOVING 
PICTURES AND ASSOCIATED AUDIO INFORMATION: SYSTEMS 

AMENDMENT 7 



1) Replace table 2-18 in subclause 2: 



Table 2-18 — Streamid assignments 



stream id 


Note 


stream coding 


1011 1100 


1 


program_streaxnjnap 


1011 1101 


2 


private__stream_l 


1011 1110 




padding_stream 


10111111 


3 


private stream_2 * 


HOxxxxx 




ISO/IEC 13818-3 or ISO/TEC 11172-3 orlSO/LEC 
13818-7 or ISO/1EC 14496-3 audio stream number 
x xxxx 


1110 xxxx 




ITU-T Rec. H.262 | ISO/LEC 13818-2 or ISO/LEC 
1 1 172-2 or ISO/IEC 14496-2 video stream number 
xxxx 


1111 0000 


3 


ECMstream 


1111 0001 


3 


EMMstream 


1111 0010 


5 


ITU-T Rec. H.222.0 | ISO/LEC 13818-1 Annex A 
or ISO/IEC 1381 8-6_DSMCC_stream 


1111 0011 


2 


ISO/ffiC_13522_stream 


1111 0100 


6 


ITU-T Rec. H.222. 1 type A 


1111 0101 


6 


ITU-T Rec. H.222.1 type B 


1111 0110 


6 


ITU-T Rec. H.222.1 type C 


1111 0111 


6 


ITU-T Rec. H.222. 1 type D 


1111 1000 


6 


ITU-T Rec. H.222.1 type E 


1111 1001 


7 


ancillary stream 


1111 1010 




ISO/IEC 14496-l_SL-packetized__stream 


1111 1011 




ISO/IEC 1 4496- lFlexMuxstream 


1111 1100... 1111 1110 




reserved data stream 


1111 1111 


4 


program_streaxn_directory 
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The notation x means that the values 'O* or T are both pennitted and results in the same stream type The stream 
number is given by the values taken by the x's. ^ in saesin 

NOTES 

1 PES packets of type program_stream_map have unique syntax specified in 2.5.4. 1 . 

2 PES packets of type private_stream_l and ISO/IEC 13552 stream follow the same PES nadrPt ,vn^v *c 
those for ITU-T Rec. H.262 | ISO/IEC 13818-2 video and KO/IEC 13818-3 aE Z^Z * ^ 

3 PES packets of type private_stream_2, ECM_streatn and EMM stream are similar to private stream 1 
except no syntax is specified after PES_packetJength field. ~ pnvate_stream_Z 

4 PES packets of type program_stream_directory have a unique syntax specified in 2.5.5. 

5 PES packets of type DSM-CC__stream have a unique syntax specified in ISO/IEC 13818-6. 

6 This stream Jd is associated with stream_type 0x09 in Table 2-29. 

7 This streamed is only used in PES packets, which carry data from a Program Stream or an ISO/IEC 1 1 1 72 I 
System Stream, in a Transport Stream (refer to 2.4.3.7). U ^ 1 1 172 1 



2) Replace table 2-26 in subclause 2: 



Table 2-26 - table 



value 


description 


0x00 


program association section 


0x01 


conditional access section fCA section) 


0x02 


I'S program map section 


0x03 


TS description section 


0x04 


ISO TEC 14496 scene description section 


0x05 


ISO IEC 14496 object descriptor section 


0x06-0x37 


ITU-T Rec. H.222.0 | ISO/EEC 13818-1 
reserved 


Ox38-Ox3F 


Defined in ISO/EEC 13818-6 


Ox40-0xFE 


User private 


OxFF 


forbidden 



3) Replace table 2-29 in subclause 2: 



Table 2-29 - 



Value 


Description 


0x00 


ITU-T | ISO/DEC Reserved 


0x01 


ISO/IEC 11172 Video 


0x02 


ITU-T Rec. H.262 | ISO/IEC 13818-2 Video or 
1 ISO/IEC 1 1 172-2 ccmstrained parameter video 
stream 


0x03 


ISO/IEC 11 172 Audio 


0x04 


ISO/TEC 13818-3 Audio 


0x05 


ITU-T Rec. H.222.0 | ISO/EEC 13818-1 
private sections 


0x06 


mtTRec. H.222.0 | ISO/EEC 13818-1 PES 
packets containing private data 


0x07 


ISO/IEC 13522 MHEG 


0x08 


ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Annex A 
DSM CC 
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0x09 


ITU-T Rec. H.222.1 ... 


OxOA 


ISO/IEC 13818-6 type A 


OxOB 


ISO/IEC 13818-6 type B 


OxOC 


ISO/IEC 13818-6 type^C 


OxOD 


ISO/IEC 13818-6 type^D 


OxOE 


ITU-T Rec. H.222.0 | ISO/IEC 13818-1 auxiliary 


OxOF 


ISO/IEC 13818-7 Audio with ADTS transport 
syntax 


0x10 


ISO/IEC 14496-2 Visual 


Oxll 


ISO/IEC 14496-3 Audio with the LATM transport 
syntax as defined in ISO/IEC 14496-3 / AMD 1 


0x12 


ISO/DEC 14496-1 SL-packetized stream or 
FlexMux stream carried in PES packets 


0x13 


ISO/EEC 14496-1 SL-packetized stream or 
FlexMux stream carried in ISO/IEC14496 sections. 


0x14 


ISO/EEC 13818-6 Synchronized Download Protocol 


0xl5-0x7F 


ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Reserved 


0x80-0xFF 


User Private 



4) Replace table 2-39 in subclause 2: 



Table 2-39 — Program and program element descriptors 



descriptor tag 


TS | 


PS 


Identification 


0 


n/a 


n/a 


Reserved 


1 


n/a 


n/a 


Reserved 


2 


x 


x 


video stream descriptor 


3 


X 


X 


audio_stream_descriptor 


4 


X 


X 


hierarchy_descriptor 


5 


X 


X 


registration_descrrptor 


6 


X 


X 


data_stream_alignment_descriptor 


7 


X 


X 


target_background_grid_descriptor 


8 


X 


X 


Video window descriptor 


9 


X 


X 


CA descriptor 


10 


X 


X 


ISO_639_language_descriptor 


11 


X 


X 


System^ clock_descriptor 


12 


X 


X 


Mulnplex_bufTer_utilizanon_descriptor 


13 


X 


X 


Copyright_descriptor 


14 


X 




Maximum_bitate_descriptor 


15 


X 


X 


Private_data_indicator_descriptor 


16 


X 


X 


Smoothing buffer descriptor 


17 


x 




STDdescriptor 


18 


X 


X 


IBPdescriptor 


19-26 


X 




Defined in ISO/EEC 13818-6 


27 


X 


X 


MPEG-4_videq_descriptor 


28 


X 


X 


MPEG-4_audio_descriptor 


29 


X 


X 


IOD_descriptor 


30 


X 




SL_descriptor 


31 


X 


X 


FMC_descriptor 


32 


X 


X 


ExternalESIDdescriptor 


33 


X 


X 


MuxC odedescrip tor 


34 


X 


X 


FrnxBufferSize_descriptor 


35 


X 




MulnplexBufTer descriptor 


36-63 


n/a 


n/a 


ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Reserved 



ITU-T Rec H.222.0 (1996)/Amd.7(199x E) 



ISO/EEC 13818-l:1996/Amd.7:199x(E) 

64-255 I n/a 1 n/a | User Private | 



5) Add after subclause 2.6.35: 



2.6.36 MPEG-4 video descriptor 

For individual ISO/IEC 14496-2 streams directly carried in PES packets, as defined in clause 2 112 the 
MPEG-4 video descriptor provides basic information for identilying the coding parameters of such visual 
elementary streams. The MPEG-4 video descriptor does not apply to ISO/EEC 14496-2 streams 
encapsulated in SL-packets and in FlexMux packets, as defined in clause 2.11.3. 



Syntax 


No. of bits 


Mnemonic 


MPEG^_video_descriptor 0 { 
descriptor__tag 
descriptorjength 
MPEG-4 visuai_profile and level 

} 


8 
8 
8 


uimsbf 
uimsbf 
uimsbf 



2.6.37 Semantic definition of fields in MPEG-4 video descriptor 

MPEG-4_videojrofile_and level - This 8-bit field shall identify the profile and level of the ISO/IEC 
14496-2 video stream. This field shall be coded with the same value as the profile_and_level indication 
field in the Visual Object Sequence Header in the associated ISO/IEC 14496-2 stream. 



2.6.38 MPEG-4 audio descriptor 

For individual ISO/IEC 14496-3 streams directly carried in PES packets, as defined in clause 2 1L2 the 
MPEG-4 audio descriptor provides basic information for identifying the coding parameters of such audio 
elementary streams. The MPEG-4 audio descriptor does not apply to ISO/EEC 14496-3 streams 
encapsulated m SL-packets and in FlexMux packets, as defined in clause 2.1 1.3. 



Syntax 


No. of bits 


Mnemonic 


MPEG-4_audio_descriptor 0 { 
descriptortag 
descriptorjength 
MPEG-4 audiojprofile and level 

} " - 


8 
8 
8 


uimsbf 
uimsbf 
uimsbf 



2.6.39 Semantic definition of fields in MPEG-4 audio descriptor 

^i^-^^^ 6 -"^ 16 ^ ' 1Wf *~ hit fcld ^ i6sDjdf y Profile and level of the ISO/DEC 
14496-3 audio stream corresponding to the following table. 

Table 2-27 - MPEG-4 j*udiojproffle_and_level assignment values 
I Value J " Description | 
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0x00 - OxOF 


Reserved - * 


0x10 
0x11 
0x12 
uxio 
0x14-0x17 


Main profile, level 1 
Main profile, level 2 ^ 
Main profile, level 3 

A/Tarn T^rnfil** If^vf* 1 4 

Reserved 


0x18 
0X19 
OxlA 
OxlB 
OxlC-OxlF 


Scalable Profile, level 1 
Scalable Profile level 2 
Scalable Profile, level 3 
Scalable Profile, level 4 
Reserved 


0x20 
0x21 
0x22-0x27 


Speech, profile, level 1 
Speech profile, level 2 
Reserved 


0x28 
0x29 
0x2A 
0x2B-0x2F 


Synthesis profile, level 1 
Synthesis profile, level 2 
Synthesis profile, level 3 
Reserved 


0x30 
0x31 
0x32 
0x33 
0x34-0x37 


High quality audio profile, level 1 
High quality audio profile, level 2 
High quality audio profile, level 3 
High quality audio profile, level 4 
Reserved 


0x38 
0x39 
Ox3A 
0x3B 
0x3C-0x3F 


Low delay audio profile, level 1 
Low delay audio profile, level 2 
Low delay audio profile, level 3 
Low delay audio profile, level 4 
Reserved 


0x40 
0x41 
0x42 - 0x47 


Natural audio profile, level 1 
Natural audio profile, level 2 
Reserved 


0x48 
0x49 
0x4A 
0x4B-0xFF 


Mobile communications profile, level 1 
Mobile communications profile, level 2 
Mobile cornmunications profile, level 3 
Reserved 



2.6.40 IOD descriptor 

The IOD descriptor encapsulates the InitialObjectDescriptor structure. An initial object descriptor allows 
access to a set of ISO/EEC 14496 streams by identifying the ES_ID values of the ISO/EEC 14496-1 scene 
description and object descriptor streams. Both the scene description stream and the object descriptor 
stream contain further information about the ISO/EEC 14496 streams that are part of the scene. See Annex 
I§ for a description of the content access procedure. The InitialObjectDescriptor is specified in sub-clause 
8.6.3 of ISO/IEC 14496-1. 

Within a Transport Stream, the IOD descriptor shall be conveyed in the descriptor loop immediately 
following the program info_length field in the Program Map Table. If a Program Stream Map is present in 
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a Program Stream, the IOD descriptor shall be conveyed in me descriptor loop immediately following the 
program _strearn_info_lerigm field in the Program Stream Map. More than one IOD descriptor may be 
associated to a program. 

NOTE - This specification does not specify how the lODJabel may be used by higher level service 
information to uniquely select one of the ISO/IEC 14496 presentations identified by multiple IOD 
descriptors. 3 y 



Syntax 


No. of bits 


Mnemonic 


IODdescriptor 0 { 

descriptor tag 
descriptorlength 
Scope_ofIOD label 
IODJabel 

InitialObj ectDescriptor Q 


8 
8 
8 
8 


uimsbf 
uimsbf 
uimsbf 
uimsbf 



2.6.41 Semantic definition of fields in IOD descriptor 

^Y'^Z 1 ^' 7^ *** fidd Spwifies me sc °P e of * e KJDJabd field. A value of 0x10 
mdicates that the IOD label is unique within the Program Stream or within the specific program in a 
Transport Stream m which the IOD descriptor is carried. A value of 0x11 indicates that the IOD label is 
unique wrfhm the Tranqwrt Stream in which the IOD descriptor is carried. All other values" of the 
S>cope_of_IOD_label field are reserved. 

IODJabel - This 8-bit field specifies the label of the IOD descriptor. 

InitialObjectDescriptor 0 - This structure is denned in sub-clause 8.6.3.1 of ISO/IEC 14496-1 



2.6.42 SL descriptor 

The SL descriptor shall be used when a single ISO/IEC 14496-1 SL-packetized stream is encapsulated in 
PES packets. The SL descriptor associates the ES JD of this SL-packetized stream to an elementary PID in 
case of a Transport Stream or to an elementary_stream_id in case of a Program Stream. Within a Transport 
Stream, the SL descriptor shall be conveyed for the corresponding elementary stream in the descriptor loop 
immediately following the ES_info_Iength field in the Program Map Table. If a Program Stream Map is 
present m a Program Stream, the SL descriptor shall be conveyed in the descriptor loop immediately 
following the elenientary_stream_mfoJength field within the Program Stream Map. 

^^ 6 ^f d St ? 3mS ™*yl*usedma Program Stream. However, only one stream id exists for 
I ti5^ P ^ t f ed StreamS - I" order to associate multiple such streams within a Program 
Stream to an ISO/EEC 14496-1 scene, FlexMux has to be used and signaled appropriately by an FMC 
descriptor. This limitation does not exist in a . Transport Stream where the SL descriptor provides 
unamb^uous mapping between an ISO/IEC 14496-1 ES_ID value and an ITU-T Rec. H.222.0 I ISO/EEC 
1 3 o l o - 1 eiementary_PID value. 



Syntax 


No. of bits 


Mnemonic 


SL_descnptor 0 { 

descriptortag 
descriptor length 
ES ID 

} 


8 
8 
16 


uimsbf 
uimsbf 
uimsbf 
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Syntax ' 


No. of bits 


Mnemonic 


External_ES JOD_descriptor 0 { 
descriptortag 
descriptorlength 
External ES ID 


8 
8 

16 


uimsbf 
uimsbf 
uimsbf 



2.6.47 Semantic definition of fields in External_ES_ID descriptor 

ExternalJESJTO - This 16-bit field assigns an ES_ID identifier, as defined in ISO/EEC 14496-1, to a 
component of a program. 

2.6.48 Muxcode descriptor 

The Muxcode descriptor conveys MuxCodeTableEntry structures as defined in subclause 11.2.4.3 of 
ISO/EEC 14496-1. MuxCodeTableEntries configure the MuxCode mode of FlexMux. 
One or more Muxcode descriptors may be associated to each elementary_PID or elementary_stream_id, 
respectively conveying an ISO/IEC 14496-1 FlexMux stream that utilizes the MuxCode mode. Within a 
Transport stream, the muxcode descriptor shall be conveyed for the corresponding elementary stream in the 
descriptor loop immediately following the ES_info_lenglh field in the Program Map Table. If a Program 
Stream Map is present in a Program Stream, the Muxcode descriptor shall be conveyed in the descriptor 
loop immediately following the elementary jstreamjnfojength field in the Program Stream Map. 
MuxCodeTableEntries may be updated with new versions. In case of such updates, the version number of 
each Program Map Table or the program_stream_map_version of each Program Stream Map, respectively, 
carrying the MuxCode descriptor in their descriptor loop shall be incremented by 1 modulo 32. 



Syntax 


No. of bits 


Mnemonic 


Muxcodedescriptor () { 
descriptor_tag 
descriptor length 

for(i = 0;i<N;i++) { 

MuxCodeTableEntry 0 

} 

} 


8 
8 


uimsbf 
uimsbf 



2.6.49 Semantics of fields in Muxcode descriptor 

MuxCodeTableEntry 0 - This structure is defined in sub-clause 1 1.2.4.3 of ISO/IEC 14496-1. 

2.6.50 FmxBufferSize descriptor 

The FmxBufferSize descriptor conveys the size of the FlexMux buffer (FB) for each SL packetized stream 
multiplexed in a FlexMux stream. 

One FmxBufferSize descriptor shall be associated to each elementary_PID or elementary_stream_id, 
respectively, conveying an ISO/IEC 14496-1 FlexMux stream. Within a Transport stream, the 
FmxBufferSize descriptor shall be conveyed for the corresponding elementary stream in the descriptor loop 
immediately following the ESJnfoJength field in the Program Map Table. If a Program Stream Map is 
present in a Program Stream, the FmxBufferSize descriptor shall be conveyed in the descriptor loop 
immediately following the elementaiy_stream_info_length field within the Program Stream Map. 
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2.6.43 Semantic definition of fields in SL descriptor 

ES_ID - This 16-bit field shall specify the identifier of an ISO/IEC 14496-1 SL-packetized stream. 

2.6.44 FMC descriptor ^ 

uSS^^^i iB fT &B IS0/IEC 14496 " 1 nexMux too1 ta been ^ ed to ^tiplex ISO/EEC 
^SScSSlS^ TZr?T * FI£XMUX Stteam «"*« ^capsulation in PES packed 

e^ h th^C k d^tr m m a F1CXMUX Stream ' Fl6XMUX «*— ' ^ identified by a single 



Syntax — 






FMC descriptor 0 { 

descriptor tag 
descriptorjength 

for (i=0; i<descriptor length; i += 3) { 

es_id 

. FlexMuxChamiel 

} 

} 


No. of bits 

8 
8 

16 
8 


Mnemonic j 

uimsbf 
uimsbf 

uimsbf 
uimsbf 



2.6.45 Semantic definition of fields in FMC descriptor 

ES_ID - This 16-bit field specifies the identifier of an ISO/ffiC 14496-1 SL-packetized stream. 
££££££ " ^ "* *" ** ^ ° f - — for this SL- 

2.6.46 External_ESJD descriptor 

^ L S sSa^. ^ deSCnPtt ° n OT ' ^ to 3 non-ISO/IEC 14496 component 

Wimni a Transport stream, the assignment of an ES ID shall be made by conveying an External F«? m 

f£ ™ Program ^^p Table. If a Program Stream MapTpresent in a LgraTTtfei 
2lS2^ ~ desfnptor shall be conveyed in the descriptor loop immediately fbltowinTS 
elementary_stream_n»fo_length field in the Program Stream Map. 7 rouowm 8 me 



Syntax 



No. of bits | Mnemonic 
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Syntax 


No. of bits 


Mnemonic 


FmxBufferSize_descriptor () { 
descriptortag 
descripcorjength 
DefaultFIexMuxBufferDescriptorO 
for (i=0; i<descriptor_lengtii; i += 4) { 
FlexMuxBufTerDescriptorO 

} 

} 


8 
8 


uimsbf 
uimsbf 



2.6.51 Semantics of fields in FmxBufferSize descriptor 

FlexMuxBuflFerDescriptorO - This descriptor specifies the FlexMux buffer size for one SL-packetized 
stream carried within the FlexMux stream. It is defined in sub-clause 1 1 J2 of ISO/IEC 14496- 1 . 

DefaultFlexMuxBufferDescriptorO - This descriptor specifies the default FlexMux buffer size for this 
FlexMux stream. It is defined in sub-clause 112 of ISO/IEC 14496-1. 

2.6.52 MultiplexBuffer descriptor 

The MultiplexBuffer descriptor conveys the size of the multiplex buffer MB m as well as the leak rate Rxq at 
which data is transferred from transport buffer TB n into buffer MB n for a specific ISO/EEC 13818-1 
program element referenced by an elementaryPED value in the Program Map Table. 

One MultiplexBuffer descriptor shall be associated to each elementary PED that contains an ISO/IEC 
14496 FlexMux stream or SL-packetized stream, including those containing ISO_IEC_14496_sections. See 
clause 2.113,9 for the definition of buffers and rates in the T-STD model for decoding of ISO/EEC 14496 
content. 

The MultiplexBuffer descriptor shall be conveyed in the descriptor loop immediately following the 
ES_info_length field in the Program Map Table. 



Syntax 


No, of bits 


Mnemonic 


MultrplexBuffer_descTTptoT 0 { 






descriptortag 


8 


uimsbf 


descriptorjengtn 


8 


uimsbf 


MB_bufTer_size 


24 


uimsbf 


TB leak rate 


24 


uimsbf 


> 







2.6.53 Semantics of fields in MultiplexBuffer descriptor 

MBJbuffer_ste - This 24-bit field shall specify the size in byte of buffer MB a of the elementary stream n 
that is associated with this descriptor. 

TB_leak_rate - This 24-bit field shall specify in units of 400 bits per second the rate at which data is 
transferred from transport buffer TBq to multiplex buffer MB n for the elementary stream n that is associated 
with this descriptor. 

4* 

6) Add after subclause 2.4.2.6: 
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2.4.2.7 T-STD extensions for carriage of ISO/IEC 14496 data 

For decoding of ISO/EEC 14496 data carried in a Transport Stream the T-STD model is extended. T-STD 
parameters for decoding of individual ISO/IEC 14496 elementary streams are defined in clause 2.11.2, 
while clause 2.1 1.3 defines T-STD extensions and parameters for decoding of ISO/EEC 14496 scenes and 
associated streams. 

7) Add after subclause 2.5.2.5: 

2.5.2.6 P-STD extensions for carriage of ISO/IEC 14496 data 

For decoding of ISO/TEC 14496 data carried in a Program Stream the P-STD model is extended. For 
decoding of individual ISO/IEC 14496 elementary streams in the P-STD see clause 2.11.2. Clause 2.11.3 
defines P-STD extensions and parameters for decoding of ISO/EEC 14496 scenes and associated streams. 

8) Changes to ciause 2.4.2.6 

Replace the text 

The delay of any data through the System Target Decoders buffers .shall 
be less than or equal to one second except for still picture video data. 

with 

Hie delay of any data through the System Target Decoder buffers shall 
be less than or equal to one second except for still picture video data and 
ISO/IEC 14496 streams. 

Add the following paragraph immediately before the Definition of overflow and underflow : 

For ISO/IEC 14496 streams, the delay is constrained by tdnQ-^i) <= 10 
second for all j, and all bytes i in access unit An(j). 

9) Changes to clause 2.5.2.3 

Replace the text 

For all Program Streams, the delay caused by system target decoder input 
buffering shall be less than or equal to one second except for still picture 
video data. 

with 

For all Program Streams, the delay caused by system target decoder input 
buffering shall be less than or equal te one second except for still picture 
video data and ISO/IEC 14496 streams. 

Replace the text 

Specifically: in the case of no still picture video data the delay is 
constrained by: 

tdaO'Xi) <= 1 sec 
else in the case of still picture video data the delay is constrained by: 

td„0>t(i)<=60sec 
for all bytes contained in access unit j. 

with 
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Specifically: in the case of no still picture video data and no ISO/TEC 
14496 stream the delay is constrained by: 

t<Uj)-t(i) <= 1 sec 
in the case of still picture video data the delay is constrained by: 

tfUD-tO) <= 60 sec 
in the case of ISO/IEC 14496 streams the delay is constrained by: 

tdnG'XO <= 10 sec 
for all bytes contained in access unit j. 



10) Add clause 2.11 immediately after clause 2.10: 



2.11 Carriage of ISO/IEC 14496 data 

2.11.1 Introduction 

An ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream may carry individual ISO/IEC 14496-2 and 14496-3 
elementary streams as well as ISO/IEC 14496-1 audiovisual scenes with its associated streams. Typically, 
the ISO/IEC 14496 streams will be elements of an ISO/EEC 13818-1 program, as defined by the PMT in a 
Transport Stream and the PSM in a Program Stream. 

For the carriage of ISO/IEC 14496 data in Transport Streams and Program Streams distinction is made 
between individual elementary streams and an ISO/IEC 14496-1 audiovisual scene with its associated 
streams. 

• For carriage of individual ISO/EEC 14496-2 and 14496-3 elementary streams only system tools from 
ITU-T Rec. H.222.0 | ISO/IEC 13818-1 are used, as defined in clause 2.1 1.2; 

• For carriage of an audiovisual ISO/IEC 14496-1 scene and associated ISO/EEC 14496 elementary 
streams, contained in ISO/IEC 14496-1 SLjacketized streams or FlexMux streams, tools from both 
ITU-T Rec. H.222.0 | ISO/IEC 13818-1 and from ISO/EEC 14496-1 are used, as defined in clause 
2.11.3. 

2.1 1 .2 Carriage of individual ISO/IEC 14496-2 and 14496-3 Elementary Streams 
in PES packets 

2.11.2.1 Introduction 

Individual ISO/EEC 14496-2 and 14496-3 elementary streams may be carried in PES packets as 
PES_packet_data_bytes. For PES packetization no specific data alignment constraints apply. For 
synchronisation PTSs and, when appropriate, DTSs are encoded in the header of the PES packet that carries 
the ISO/IEC 14496 elementary stream data; for PTS and DTS encoding the same constraints apply as for 
ISO/IEC 13818 elementary streams. See table 2-62 for an overview of how to carry individual ISO/IEC 
14496 streams within an ISO/TEC ITU-T Rec. H.222.0 | 13818-1 stream. 

Table 2-62 - Carriage of individual ISO/IEC 14496 streams in ITU-T Rec. H.222.0 1 ISO/IEC 13818-1 



ISO/IEC 14496-2 visual 


Carriage in PES packets 


Stream type ~ 0x10 


Stream id = * 1 1 10 xxxx* 


ISO/IEC 14496-3 audio 


Carriage in PES packets 


Streamtype = 0x1 1 


Streamid = ' 1 lOx xxxx' 



If a PTS or DTS is present in the PES packet header it shall refer to the visual object that follows either the 
first VOP start code or the first still texture object startcode that commences in the PES packet Each 
ISO/IEC 14496-2 video stream carried by ITU-T Rec. H.222.0 | ISO/DEC 13818-1 shall contain the 
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information required to decode the ISO/EEC 14496-2 video stream; consequently the stream shall contain 
Visual Object Sequence Headers, Visual Object Headers and Video Object Layer Headers. 

In case of an ISO/IEC 14496-3 elementary stream, before PES packetization the elementary stream data 
shall be first encapsulated in the LATM transport syntax defined in ISO/EEC 14496-3 / AMD 1. If a PTS is 
present in the PES packet header it shall refer to the first audio frame that follows the first syncword that 
commences in the payload of the PES packet 

Carriage of individual ISO/IEC 14496 elementary streams in PES packets shall be identified by appropriate 
stream id and stream type values, indicating the use of ISO/EEC 14496-2 Visual or 14496-3 Audio. In 
addition, such carriage shall be signaled by the MPEG-4 video descriptor or MPEG-4_audio descriptor, 
respectively. These descriptors shall be conveyed in the descriptor loop for the respective elementary 
stream entry in the Program Map Table in case of a Transport Stream or in the Program Stream Map, when 
present, in case of a Program Stream. ITU-T Rec. H.222.0 | ISO/IEC 13818-1 does not specify presentation 
of ISO/IEC 14496 elementary streams in the context of a program. 

2.1 1 .2.2 STD extensions for individual ISO/IEC 14496 elementary streams 

The T-STD model includes a transport buffer TB n and a multiplex buffer B n prior to decoding of each 
individual ISO/EEC 14496 elementary stream n. Note that in the T-STD the single multiplex buffer B„ is 
also applied for ISO/IEC 14496-2 video, as indicated in Figure 2-4, instead' of the approach with two 
buffers MB* and EB n used for ISO/IEC 13818-2 video in the T-STD. For buffers TBn and B n and the rate 
Rxn between TB n and Bn the following coristraints apply. 




| »-th byte of f 
I Transport Stream j 



\ 3-th access unit J [ k-th composition unit j 





TB, 




Bi 


— i"' 


M — 







Video 




U o, -O 



C n (k) 
Audio te (k) 



Figure 2-4 - T-STD model extensions for individual ISO/DEC 14496 elementary streams 



In case of carriage of a ISO/IEC 14496-2 stream: 
Size BSa of Buffer B n : 

BS,, = BS^ + BSoh + VB V m [profae,level] 

where BSoh, packet overhead buffering, is defined as: 

BS^ (1/750) seconds x maxfR^Jprofile^evel], 2 000 000 bit/second} 
and BSnnHo additional multiplex buffering, is defined as: 

BS mnx" 0.004 seconds x rriaxjfRn^roflleJevel], 2 000 000 bit/second} 
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RateRXo : 

RXn = 1.2 x Rn^[profIle,level] 

Where VBV^fprofileJevel] and R^fprofile^evel} are defined in ISO/TEC 14496-2 for each 
profile and level. For profiles and levels for which no VBVn^ value is specified, the size of Bn 
and the rate Rx* are user defined. 

In case of carriage of an ISO/IEC 14496-3 stream: 

Size BSn of Buffer Bn for ISO/EEC 14496-3 AAC audio as defined in this Specification, 

elseBS a -BS mu3l +BS dec + BS<> h =3584 bytes 

In this case the size of the access unit decoding buffer BS^eo and the PES packet overhead 
buffer BSoh are constrained by 

BSdec + BS^ 2848 bytes 

A portion (736 bytes) of the 3584 byte buffer is allocated for buffering to allow multiplexing. 
The rest, 2848 bytes, are shared for access unit buffering BS<ko BS^ and additional 
multiplexing. 

Rate Rxn for ISO/IEC 14496-3 AAC audio as defined in this Specification, 
else Rxa = 2 000 000 bit/sec. 

The P-STD model includes a multiplex buffer B n prior to decoding of each individual ISO/IEC 14496 
elementary stream n. The size BS B of buffer B n in the P-STD is defined by the P-STD_buffer_size field in 
the PES packet header. 



2.1 1.3 Carriage of audiovisual ISO/IEC 14496-1 scenes and associated ISO/IEC 
14496 streams 



2.11.3.1 Introduction 

This clause describes the encapsulation and signaling when an audiovisual scene represented by ISO/EEC 
14496 data is carried in an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Program Stream or Transport Stream. 
ISO/IEC 14496 content consists of the initial object descriptor and a variable number of streams such as 
object descriptor streams, scene description streams (carrying either BIFS-Command or BEFS-Anim access 
units), EPMP streams, OCI streams and audio-visual streams. Each of the ISO/DEC 14496 streams shall be 
contained in an SL-packetized stream and may optionally be multiplexed into a FlexMux stream, bom 
defined in ISO/EEC 14496-1. For carriage in ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Program Stream or 
Transport Stream, these SL-packetized streams and FlexMux streams shall contain encoded Object Clock 
Reference (OCR) and FlexMux Clock Reference (FCR) fields as specified in 2.11.3.4 and in 2.11.3.5, 
respectively. The SL-packetized streams or FlexMux streams are then encapsulated either in PES packets 
or in ISO_IEC_14496_sections prior to Transport Stream packerization and multiplexing or Program 
Stream multiplexing. ISO_IEC_ 1 4496_sections are built on the long format of H.222.0 | ISO/IEC 13818-1 
sections. 

2.1 1 .3.2 Assignment of ESJD values 

An ISO/IEC 14496-1 scene carried over an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream may associate a . 
number of ISO/IEC 14496, ISO/IEC 13818 and other streams by the use of the ESJCD parameter. The 
scene and the associated streams may be carried over the same ITU-T Rec. H.222.0 | ISO/IEC 13818-1 
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stream, but a scene may also reference streams carried elsewhere, for example over an DP network. How to 
identify such other means is not defined in this Specification. 

ISO/TEC 14496-1 defines name scoping rules for identifiers. These rules allow the same ES ID value to be 
used for two different streams within ISO/IEC 14496 content When one or multiple ISO/IEC 14496-1 
scenes are carried in a ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Program, duplicate ESJD values shall not 
occur within the program such that each ISO/TEC 14496 SL-packetized stream or ISO/IEC 14496-1 
Flexmux channel has a unique ES ED value in the program. 



2. 1 1 .3.3 Timing of ISO/IEC 1 4496 scenes and associated streams 

When carried over an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream, the object time base of each ISO/EEC 
14496 stream shall be locked to the ITU-T Rec. H.222.0 | ISO/DEC 13818-1 STC, that is : 
If X(t) « f^t) / f^t), then 

the value of X(t) shall be constant at any time t 
Where fsJt) denotes the intended ftequency of the STC at time t, i.e. 27 000 000 Hz 
fcfejcct(t) denotes the frequency of the object time base at time t 

The object time base of ISO/IEC 14496 streams carried over an ITU-T Rec. H.222.0 I ISO/IEC 13818-1 
stream is conveyed as follows : 

- The object time base of an SL-packetized stream carried in PES packets without the use of the 
FlexMux shall be conveyed by coded OCRs in the SL packet header of that stream. See clause 
2.11.3.4. 

- The object time base of SL-packetized streams carried in PES packets within a FlexMux stream shall 
be conveyed by FCRs in mat FlexMux stream. See clause 2.11.3.5. Consequently, all ISO/IEC 14496 
streams contained within the same FlexMux stream share the same object time base. 

" ^/^ JCCt timC baSC ° f 211 SL "P acketized stTGam carried in sections shall be conveyed by another 
.ISCMEC 14496 stream within the Transport Stream or Program Stream as indicated by the 
OCRJBSJD field in the ES descriptor for that stream 

The following constraints shall apply for encoding of OCRs and FCRs in SL-packetized streams and 
FlexMux streams carried over an ITU-T Rec. H.222,0 | ISO/DEC 13818-1 stream : 

- The OCRs and FCRs in each SL-packetized stream and each FlexMux stream associated to the same 
scene shall have the same resolution. 

- Hie resolution of OCRs and FCRs for a scene, shall have a value smaller than or equal to 90 000 
Hz. 

- The ratio (f^(t) / 300) / r^, shall be an integer value larger than or equal to one. Consequently the 
resolution of the OCR and FCR syntax elements may only take values such as 90 000 Hz, 45 000 Hz. 
30 000 Hz, 22 500 Hz, 18 000 Hz, etc. 

Within the above constraints and the ISO/IEC 14496-1 constraint that the resolution ^ shall represent an 
integer number of cycles per second, can be selected as appropriate for the scene. 

The ISO/IEC 14496 time stamps coded in the SL packet header shall refer to instants of the object time 
base of the stream carried in the SL packet The resolution of each such time stamp shall be of a factor 2 k 
smaller than the resolution of the OCRs or FCRs associated to the stream, with k a positive integer larger 
than or equal to zero. To achieve the same wrap around, the length of the time stamp fields, 
TmeStampLength, shall be k bit smaller man die length of the OCR or FCR field, OCRLength and 
stamp^ respectively. Hence for each stream the following conditions shall apply for encoding of time 

- TimeStampResolution = (OCRResolution or FCRResolution respectively) / 2 k , with k a positive 
integer larger than or equal to zero. ISO/IEC 14496-1 requires TimeStampResolution to represent an 
integer number of cycles per second. ^ 

- TimeStampLength = OCRLength or FCRLength respectively - k. 
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The relationship between a value of the STC and the corresponding value of the object time base of a 
stream is established by associating PTS fields in PES packet headers with the OCR or FCR in SL packet 
headers and FlexMux Stream packets, respectively, as specified in 2.1 1.3.6 and 2.1 1.3.7. 

2.11 .3.4 Delivery timing of SL-packetized streams 

To carry ISO/DEC 14496 content in an ITU-T Rec. H.222.0 | ISO/EEC 13818-1 stream, ISO/IEC 14496-1 
SL-packetized streams are used. In each SL-packetized stream carried in a PES packet without the use of 
FlexMux, the objectClockReference field* shall be encoded as follows: 

1 . An objectClockReference (OCR) field shall be present in the first SL packet header of a SL-packetized 
stream. 

2. The SL-packetized stream shall be constructed such that the time interval between the bytes containing 
the last bit of successive OCR fields shall be less than or equal to 0,7 second. Thus: 

|t(i") -tOI <-<V7s 

for all i* and i" where i* and i" are the indexes of the bytes containing the last bit of 
consecutive OCR fields in the FlexMux stream. 

If an objectClockReference is encoded in an SL packet header, also the instantBitrate field shall be coded 

2. 1 1 .3.5 Delivery timing of FlexMux streams 

Next to SL-packetized streams also the ISO/DEC 14496-1 FlexMux tool may be used to carry ISO/IEC 
14496 content in ITU-T Rec. H.222.0 | ISO/IEC 13818-1 streams. The payload of FlexMux packets shall 
consist of SL packets as specified in ISO/IEC 14496-1. In each FlexMux stream carried in an ITU-T Rec. 

H. 222.0 | ISO/EEC 13818-1 stream the fmxCIockReference field shall be encoded as follows : 

I . An fmxCIockReference (FCR) field shall be present in the first FlexMux packet of a FlexMux stream 

2. The FlexMux stream shall be constructed such that the time interval between the bytes containing the 
last bit of successive FCR fields shall be less man or equal to 0,7 second. Thus : 

|t(i") -tOI < = 0,7s 

for all i 1 and i" where i' and i" are the indexes of the bytes containing the last bit of 
consecutive FCR fields in the FlexMux stream. 

3. All ISO/IEC 14496 time stamps within the SL-packetized streams carried within a FlexMux stream 
shall refer to instants of the object time base conveyed by the FCR fields in the FlexMux stream The 
SL-packetized streams carried in FlexMux packets need not carry OCR fields. If OCR fields are 
present, they may be ignored. 

2. 1 1 .3.6 Carriage of SL-packetized streams in PES packets 

A single ISO/IEC 14496-1 SL-packetized stream may be mapped into a single PES stream One and only 
one SL packet from an SL-packetized stream shall constitute the payload of one PES packet. PES packets 
that carry an SL-packetized stream shall be identified by streamid = OxFA in the PES packet header. 

When an OCR field is coded in the SL packet header, a PTS shall be encoded in the header of the PES 
packet that carries such SL packet header. This PTS shall be encoded with the 33 bit value of the 90 kHz 
portion of the STC that corresponds to the value of the object time base at the instant in time indicated by 
the OCR. 

The ES ID associated to the SL-packetized stream shall be signaled by an SL descriptor as specified in 
2.6.46. 
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2.1 1 .3.7 Carriage of FlexMux streams in PES packets 

PES packets with a payload consisting of FlexMux packets shall be identified by streamid - OxFB in the 
PES packet header. An integer number of FlexMux packets shall constitute the payload of one PES packet, 
Le., the payload of a PES packet carrying a FlexMux stream shall start with a FlexMux packet header and 
shall end with the last byte of a FlexMux packet 

If an mixClockReference (FCR) field is encoded in one of the FlexMux packets contained in a PES packet, 
then a PTS shall be encoded in the header of the PES packet that contains such FlexMux packet This PTS 
shall be encoded with the 33 bit value of the 90 kHz portion of the STC that corresponds to the value of the 
object time base of the FlexMux stream at the instant in time indicated by the FCR. In case multiple 
FlexMux packets with an encoded FCR field are contained in a PES packet, the PTS shall correspond to the 
time indicated by the FCR in the first such FlexMux packet encountered in the payload of the PES packet 

The ESJDs associated to each SL-packetized stream conveyed in the FlexMux stream shall be signaled by 
an FMC descriptor as specified in 2.6.38. 

2.1 1 .3.8 Carriage of SL packets and FlexMux packets in sections 

For transport of ISO/IEC 14496 content in sections, ISO_IEC_ 1 4496 sections are defined. Only SL- 
packetized object descriptor streams and scene description streams shall use ISO_IEC_14496_sections. A 
single ISO_EEC_14496_section shall contain either an entire SL packet of an SL-packetized stream or an 
integer number of FlexMux packets each carrying an SL packet of the same ISO/IEC 14496-1 elementary 
stream. 

Table 2-63 shows the syntax of ISO_IEC__14496__sections defined to convey ISO/IEC 14496-1 elementary 
streams, qualified by the table id as either object descriptor or scene description stream data. Object 
descriptor stream data consists of an Object Descriptor Table that comprises a number of object descriptors. 
The Object Descriptor Table may be transmitted in multiple ISO _EEC_ 1 4496_sections. Scene description 
data consists of a Scene Desertion Table that may comprise a number of BIFS commands. The Scene 
Description Table may be transmitted in multiple ISOJEC_14496_sections. It is not required that a 
complete table be received in order to process its payload. However, the payload of sections shall be 
processed in the correct order, as indicated by the value of the section number field in the 
ISOIEC 14496_section header bytes. 
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Table 2-63 — Section syntax for transport of ISO/IE CI 4496 streams 



Syntax 



No. of bits Mnemonic 



ISO_ffiC - 14496_section() { 

table_id " 

section_syntax_indicator 

private_indicator 

reserved 

ISO_IEC_14496_sectionJength 

tableidextension 

reserved 

version_number 

current next_indicator 

sectionnumber 

last_sectio n_num ber 

if (PMT_has_SL_descriptor(ciiiTeiit_PID)) { 
SL_Packet() 

} 

else if (PMT_has_FMC_descriptor(cxiirent_PID)) { 
for (i=l;i<N;i++) 

FlexMuxPacketO 

} 

else { 

for (i=l;i<N;i++) 
reserved 

} 

CRC 32 



8 uimsbf 



1 
1 

2 



bslbf 
bslbf 
bslbf 
12 uimsbf 
16 nimsbf 
bslbf 
uimsbf 
bslbf 
8 uimsbf 
8 uimsbf 



2 
5 
1 



8 bslbf 
32 rpchof 
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"^SdSI" ™* f** bC ** to ' 0x04 ' OT 'O* 05 ' m case ™ ISO EEC 14496 section. A value 

of 0x04' indicates an ISO_IEC_14496_ S cene_desc^tion_ S ection that carries anlsO/nfC 14496-Y scene 

section syntax indicator - This 1-bit field shall be set to ' 1 '. 

private Jndicator - This 1-bit field shall not be specified by this Specification 

ISO IEC_14496_section_Iength - This 12-bit field shall specify the number of remaining bytes in the 
^^^"r 8 , the ISO mC_14496_secnonJength field up TT^^l oft 
ISO IEC 14496 section. The value of this field shall not exceed 4093 (OxFFD) 

Sfee by ST«° n " 16 " bit fieU ^ Mt bC by SP*®"*"* ^ use and value are 

S e °r7e^T 5 " bi l? ld , *S VerSi0n aUmber ° f * e "*« Descriptor Table or 

Scene Description Table respectively. The version number shaU be incremented by 1 modulo 32 with each 
new version of the table. Version control is at the discretion of the application. "witneacn 

current next indicator - This 1-bit field shall be set to 1. 

sectionjiumber - This 8-bit field shall represent the number of the ISO IEC 14496 section The 
EST?!*?!, 0 ! ^ ISO -^ C - 14496 -^- of the Object D«^SE-: £ Scene 

w^rdd^^ 

^ eCt t n - n, r^ bCr " ? W t 8 J it fieU number ° fthe ^on of the Object Descriptor 

Table or Scene Description Table of which this section is a part P 

P ^ b ^ S ^- d J !S ^ tor( r ,rrent - PID) " a pseudo that shall be true if an SL descriptor is 

™ ito'mC^irsection. ^ ^ ** *" 138 ** 

SL_PacketO - a sync layer packet as specified in subclause 10.2.2 of ISO/IEC 14496-1. 
PM MT- J T C r deSCript ? r(CUrrent - PID) " a P seudo ^tion that shall be true if an FMC descriptor is 

c^^So^mJr^^ 6 ^ Map TaWe for 1,16 ISO/ffiC 13818 - 1 el — < 

FlexMuxPacketO - a FlexMux packet as specified in subclause 11.2.4 of ISO/TEC 14496-1. 

32 ~ b f fiCld f*l 1116 vaIue a zero ^« of the registers in the 

ISO mclSS JectiT" ° f ^ ^ H 222 -° ' BO/IEC 13818 - 1 Processing the entire 

2. 1 1 .3.9 T-STD extensions 

2.11.3.9. 1 T-STD Model for 14496 content 

Figure 2-5 shows extensions of the Transport System Target Decoder for delivery of ISO/EC 14496 
program elements encapsulated in ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Transport Streams. 
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I FlexMux buffer model ' 
I (see ISO/IEC 14496-1) I 



, l-th byte of I 
1 Transport Stream \ 
I 









W„0) 


















' MBr> 









Figure 2-5 - T-STD model for ISO/EEC 14496 content 
The following notation is used in Figure 2-5 and its description : 
TB n is the transport buffer. 

MB n is the multiplex buffer for FlexMux stream n or for SL-packetized stream n. 
FB np is the FlexMux buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DBnp Is the decoder buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DBn is the decoder buffer for elementary stream n. 

is the decoder for the elementary stream in FlexMux channel p of FlexMux stream n 
D n is the decoder for elementary stream n. 

RXq is the rate at which data are removed from TB^ 
RbXa is the rate at which data are removed from MBn. 

Anp(j) is the j A access unit in elementary stream in FlexMux channel p of FlexMux stream n. A^Q) is 

indexed in decoding order. 
A^Q) is the j* access unit in elementary stream n. A n (j)is indexed in decoding order. 
TdnpG) is the decoding time, measured in seconds, in the system target decoder of the jth access unit in 

elementary stream in FlexMux channel p of FlexMux stream n. 
Tdn(j) is the decoding time, measured in seconds, in the system target decoder of die jth access unit in 

elementary stream n. 

Cop(k) is the k* composition unit in elementary stream in FlexMux channel p of FlexMux stream n. 

Cnp(k) results from decoding A^G')- CnpOO » indexed in composition order. 
Co(k) is the k* composition unit in elementary stream el Cn(k) results from decoding A^O). Ca(k) is 

indexed in composition order. 
tanpCk) is the composition time, measured in seconds, in the system target decoder of the k* composition 

unit in elementary stream in FlexMux channel p of FlexMux stream n. 
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t(0 ^ ^ ^ at which the i* byte of the Transport Stream enters the system target 

2.11.3.9.2 Processing of FlexMux streams 

Complete Transport Stream packets containing data from FlexMux stream n are passed to the transoort 
buffer for FlexMux stream n, TB^ The size of TB„ is fixed at 512 bytes All b^lhat enter TO^ 
removed fr^TB, at a rate specified by the TBJeak rate field ^^^ ^0 

^ ^ " ^ - - ^ m buff- TB,, rate Rx, * ^SEZ 
Transport Stream packets are not delivered to MB n . duplicate 

MlTInnZT^ m i PES PB S^ * e PES hcadCT ^ Payl°ad data bytes are delivered to buffer 

ir??lo S^ 8 ^ ^ l 0t «ter MB,,, and may be used to cJ5 the system. In cieof 
SX S ^ ?^ l!^ DS ' ^ header > P^oad and CRC32 data bytes are delivered to 

buffer MB^aD other byte do >not enter MB, and rr^y be used to «mtxol the system. In^er case tS^e 
of MB n shall be specified by the MB_buffer_si2e field in the MultiplexBufFer descriptor. 
The RexMux Stream packet bytes in buffer MB. are all delivered to their associated FlexMux buffer at the 
rate specxfied by the field fmxRate encoded in the FlexMux stream and in compHance^the n"S£ 

Srch ntf ^ r 1, 2 9 ° fISO/IEC 1449 ^" Only FlexMux packet payload Sta bles" 
FlexMux channel p of FlexMux stream n enter buffer FB^ FlexMux packet header bytes in FlexMu^ 

SL^T^? ?t ? bC ?P hcable for 311 FIexMux Packets in the stream immediately fotowinTthe 
^"wStfet^^ ^7 * nC * «— tered FlexMux Clock RexeTce^e 
the PF^T ^ i 31^^ Stream ^ P resent m MBa, no data is removed from MB. Bytes from 
the PES packet header or from the ISO_IEC_14496_ S ection header that immediately precede aFIexMuT 

IM KTZ^Zr 1 ^™^ - d aDd -y be used to control the s^T^tL fronts 

ISO ffiC_14496_section CRC32 fields that irnmediately follow the last HexMux Stream rSeT^ the 

2"5JZ ta'TTS ^Trf 7 md -y be used to of me 

data. Bytes from the FlexMux Clock Reference channel are instantaneously removed and d.srarrL 3 

SLn KO/IEC 14496 ° bjeCt ti-e base to the STC. WhenTe^t " ptsTSet" r 

uponletvSg M^. P Payl ° ad ° fstrcam n CTter FlexMux demultiplexer instantaneously 
2.11.3.9.3 Definition of FlexMux Buffer, FB^ 

FmxB^tTi P °f * 5°^?* ^ ^ * e Size ° f nexM ^ buffer F Bnp is defined using the 
£r£^ ^cnptor. ^^Mux packet payload bytes are transferred from buffer FB ro to decoder 

52 fl^Z ^ C ° n T^ e t C F1CXMUX ^ model defeed in clause 1 1.2.9 of ISO/ffiC iJSET 
Si ^ byt ! S m F1CXMUX ChanDCl P of RexMux stream n enter buffer DB™ The SL packet 

h«^bytes m FlexMux channel p of FlexMux stream n are discarded and may be u^ to contTt 



2.11.3.9.4 Processing of SL-packetized streams 



^^Z^jTT n Jt***"! packets containing data from SL-packetized stream n are passed to the 
, StrCam ^ TB„. All bytes that enter TB n are removed aTa mte R^ 

rIS?x i.™. ™- leak f^1 field i n * e MultiplexBufFer descriptor. When there is no data in buffer ST 
rate Rx, is equal to zero. Duplicate Transport Stream packets are not delivered to MB.. 

M^alfor^f^. PES P "5V ,B PES P3ckct header - d Payload data bytes are delivered to buffer 
cSLfmTsO mr fjS? ™» d ° ^ MB^ and may be used to control the system. In caseTf 
carnage in ISOJEC_14496_ S e*ion S , the section header, payload and CRC32 data bytes are delivered to 
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Figure 2-5 - T-STD model for ISO/IEC 14496 content 

The following notation is used in Figure 2-5 and its description : 
TB n is the transport buffer. 

MB n is the multiplex buffer for FlexMux stream n or for SL-packetized stream n. 
FBnp is the FlexMux buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DBnp Is the decoder buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DBn is the decoder buffer for elementary stream n. 

D„p is the decoder for the elementary stream in FlexMux channel p of FlexMux stream n 

D a is the decoder for elementary stream n. 

RXg is the rate at which data are removed from TBn. 

RbXa is the rate at which data are removed from MB^. 

AapQ) is the j A access unit in elementary stream in FlexMux channel p of FlexMux stream n. A^Q) is 

indexed in decoding order. 
An(j) is the j* access unit in elementary stream n. A n (j)is indexed in decoding order. 
Tdnp(j) is the decoding time, measured in seconds, in the system target decoder of the jth access unit in 

elementary stream in FlexMux channel p of FlexMux stream n. 
Tda(j) is the decoding time, measured in seconds, in the system target decoder of the jth access unit in 

elementary stream n. 

Cnp(k:) is the k A composition unit in elementary stream in FlexMux channel p of FlexMux stream n. 

Cnp(k) results from decoding A*p(j). C^k) is indexed in composition order. 
C„(k) is the k* composition unit in elementary stream n. CJJk) results from decoding A^Q). Cn(k) is 

indexed in composition order. 
tc,tp(k) is the composition time, measured in seconds, in the system target decoder of die k A composition 

unit in elementary stream in FlexMux channel p of FlexMux stream n. 
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tcQc) is the composition time, measured in seconds, in the system target decoder of die k* composition 
unit in elementary stream n. 

hidicates the time in seconds at which the i* byte of the Transport Stream enters the system target 



t(0 



decoder. 

2. 1 1.3.9.2 Processing of FlexMux streams 

Complete Transport Stream packets containing data from FlexMux stream n are passed to the transport 
buffer for FlexMux stream n, TB,,. The size of TB„ is fixed at 512 bytes. All bytes that enter TB 
removed from TB„ at a rate Rx», specified by the TBJeakjate field in the MultiplexBuffer descriptor 
associated with FlexMux stream n. When there is no data in buffer TB n , rate Rx» is equal to zero. Duplicate 
Transport Stream packets are not delivered to MB,,. 

hi case of carriage in PES packets, the PES packet header and payload data bytes are delivered to buffer 
MB^; all other bytes leaving TB„ do not enter MB., and may be used to control the system. In case of 
^ g x« ISO -^C l4496_sections, the section header, payload and CRC32 data bytes are delivered to 
™ v.N^ otfaer *y** do not eater MB, and may be used to control the system. In either case, the size 
of MB„ shall be specified by the MB buffer size field in the MultiplexBuffer descriptor. 

The FlexMux Stream packet bytes in buffer MB. are all delivered to their associated FlexMux buffer at the 
rate specified by the field fmxRate encoded in the FlexMux stream and in compliance with the FlexMux 
buffer model defined m clause 1 1.2.9 of ISO/IEC 14496-1. Only FlexMux packet payload data bytes in 
FlexMux channel p of FlexMux stream n enter buffer FB^ FlexMux packet header bytes in FlexMux 
channel p of FlexMux stream n are discarded and may be used to control the system. The rate specified by 
the finxRate field shall be applicable for all FlexMux packets in the stream immediately following the 
FlexMux Clock Reference channel packet up to the next encountered FlexMux Clock Reference channel 
packet When there is no FlexMux stream data present in MB D> no data is removed from MB„. Bytes from 
the PES packet header or from the ISO_IEC_14496_section header that immediately precede a FlexMux 
^rl^ 6 ^^ ntaneousl y removed and discarded and may be used to control the system. Bytes from the 
ISOJEC 14496 section CRC32 fields that immediately follow the last FlexMux Stream packet in the 
section payload are removed instantaneously and discarded and may be used to verify the integrity of the 
data. Bytes from the FlexMux Clock Reference channel are instantaneously removed and discarded and 
may be used to lock the ISO/IEC 14496 object time base to the STC. When there is no PES packet or 
section payload data bytes, respectively present in MB^ no data is removed from MB„. All data that enters 
MB„ eaves it All PES packet payload bytes of stream n enter the FlexMux demultiplexer instantaneously 
upon leaving MB n . y 

2. 11.3.9.3 Definition of FlexMux Buffer, FB„ P 

For ^ch channel p of a FlexMux stream n, the size of FlexMux buffer FB_, is defined using the 
FrnxBuflferSize descriptor. FlexMux packet payload bytes are transferred from buffer FB^ to decoder 
buffer DB^ m compliance with the FlexMux buffer model defined in clause 1 1.2.9 of ISO/IEC 14496-1 
Only SL packet payload bytes in FlexMux channel p of FlexMux stream n enter buffer DB™ The SL packet 
header bytes in FlexMux channel p of FlexMux stream n are discarded and may be used to control the 
system. 



2.11.3.9.4 Processing of SL-packetized streams 

Complete Transport Stream packets containing data from SL-packetized stream n are passed to the 
transport buffer for SL-packetized stream n, TB». All bytes that enter TB„ are removed at a rate Rx^ 
specified by the TB leak rate field in the MultiplexBuffer descriptor. When there is no data in buffer TbT 
rate Rx» is equal to zero. Duplicate Transport Stream packets are not delivered to MB,,. 

Incase of carriage in PES packets, the PES packet header and payload data bytes are delivered to buffer 
■ ° t ?"^ eS leavm 8 TB. do not enter MB^ and may be used to control the system. In case of 
carnage in ISO_ffiC_14496_sections, the section header, payload and CRC32 data bytes are delivered to 
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buffer MB a ; all other bytes do not enter MBn and may be used to control the system. In either case the size 
of MB n is specified by the MB_buffer_size field in the MultiplexBuffer descriptor. 

The SL-packetized stream bytes in buffer MB n are all delivered to the decoder buffer DBn at the rate 
specified by the field instantBitRate encoded in the SL-packetized stream and in compliance with the 
System Decoder Model defined in clause 7.4 of ISO/EEC 14406-1. The rate specified by the instantBitRate 
field shall be applicable for all data bytes in the SL-packetized stream immediately following the 
instantBitRate field in the SL packet header up to the next encountered instantBitRate field. If there are no 
SL-packetized stream bytes in MB^ no bytes are removed from MBn. Bytes from the PES packet header or 
from the IS 0_IEC_ 1 449 6_section header that immediately precede a SL packet header are instantaneously 
removed and discarded and may be used to control the system. Bytes from the ISO_IEC_14496_section 
CRC32 fields that immediately follow the last SL packet payload byte in the section are removed 
instantaneously and discarded and may be used to verify the integrity of the data. When there are no PES 
packet or section payload data bytes, respectively present in MB n> no data is removed from MB n . All data 
that enters MB B leaves it. All PES packet payload bytes of stream n enter buffer DB* instantaneously upon 
leaving MB m with the exception of the SL packet headers. Bytes from the SL packet headers do not enter 
DB a and may be used to control the system. The size of decoder buffer DB n is given by the bufferSizeDB 
of the DecoderConfigDescriptor defined in ISO/EEC 14496-1. 

2. 1 1.3.9.5 Buffer management 

Transport streams shall be constructed so that conditions defined in this subclause are satisfied. 

TB n shall not overflow and shall be empty at least once every second. MB„ shall not overflow. FB^p shall 
not overflow. DB np and DB a shall neither underflow nor overflow. Underflow of DBn P occurs when one or 
more bytes of an access unit are not present in DBnp at the decoding time associated with this access unit. 
Underflow of DB a occurs when one or more bytes of an access unit are not present in DB n at the decoding 
time associated with this access unit. 

2.1 1 .3.1 0 Carriage within a Transport Stream 
2.1 1.3.10.1 Overview 

A Transport Stream may contain one or more programs, each described by a Program Map Table. ISO/IEC 
14496 content can be conveyed in addition to the already defined stream types for such a program. 
Elements of the ISO/TEC 14496 content may be conveyed in one or more ISO/TEC 13818-1 program 
elements referenced by a unique PED value within a Transport Stream. As a special case, it is possible that a 
program within a Transport Stream consists only of ISO/EEC 14496 program elements. ISO/EEC 14496 
content associated to a program and carried in the Transport Stream shall be referenced in the Program Map 
Table of that program. An initial object descriptor shall be used to define an ISO/DEC 14496-1 scene; the 
use of this descriptor is specified in clause 2.11.3.10.2. 

Carriage of ISO/IEC 14496 content in a PED is signaled by a streamjype value of 0x12 or 0x13 in the 
Program Map Table in association with that PID value. A value of 0x12 indicates carriage in PES packets. 
The stream ed field in the PES packet header signals whether the PES packet contains a single SL packet or 
a number of FlexMux packets. A stream type value of 0x13 in the Program Map Table indicates that the 
program element carries an object descriptor stream or a BEFS-Command stream contained in sections. In 
this case the table_id in the section header indicates whether an object descriptor stream is carried in the 
sections or a BEFS-Command stream. See also table 2-64. The section contains either a single SL packet or 
a number of FlexMux packets, as indicated by the presence of an SL descriptor or a FMC descriptor 
respectively in the descriptor loop of the Program Map Table for the ISO/IEC 13818-1 program element 
that carries the sections. When ISO/IEC 14496 content is carried, the SL descriptor and the FMC descriptor 
shall specify the ESID for each encapsulated ISO/IEC 14496 stream. When the assignment of ESID 
values changes, the Program Map Table shall be updated and the version_number of the PMT shall be 
incremented by 1 modulo 32. An example of a content access procedure for ISO/DEC 14496 program 
components within a Transport Stream is given in Annex R Carriage of ISO/EEC 14496 scenes in ITU-T 
Rec. H.222.0 | ISO/IEC 13818-1. 
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Table 2-64 - ISO/EEC defined options for carriage of an ISO/DEC 14496 scene and 

in ITU-T Rec. HJ22.0 | ISO/EEC 13818-1 



associated streams 



ISO/IEC 14496-1 
object descriptor 
streams 


Encapsulation in 
SL packets 


Carriage in PES 
packets 


Stream type = 
0x12 


Stream id = * 1 1 1 1 
1010' ~ 


Carriage in 
ISO_IEC_ 
14496 sections 


Stream type = 
0x13 


Tableid = 0x05 


Encapsulation in 
SL packets 
followed by 
Multiplex into 
FlexMux packets 


Carriage in PES 
packets 


Stream type = 
0x12 


Stream id = '1111 

loir 


Carriage in 

ISOJDEC_ 

14496_sections 


Stream type — 
0x13 


Table id = 0x05 


ISO/IEC 14496-1 
scene description 
streams 


Encapsulation in 
SL packets 


Carriage in PES 
packets 


Stream type - 
0x12 


Stream id = '1111 
1010' 


Carriage in 

ISO_IEC 

14496__sections 


Stream type ~ 
0x13 


Tablejd = 0x04 


Encapsulation in 
SL packets 
followed by 
Multiplex into 
FlexMux packets 


Carriage in PES 
packets 


Stream type = 
0x12 


Stream id = 
101T 


Carriage in 

ISO_IEC_ 

1 4496 sections 


Stream type — 
0x13 


Tableid = 0x04 


All other ISO/DEC 


Encapsulation in 
SL packets 


Carriage in PES 
packets 


Stream type = 
0x12 


Stream id = *1111 
1010* 


14496 streams 


Encapsulation in 
SL packets 
followed by 
Multiplex into 
FlexMux packets 


Carriage in PES 
packets 


Stream type — 
0x12 


Stream id = '1111 
1011' 



2.11.3.10.2 Initial Object Descriptor 

In case of carriage of an ISO/IEC 14496-1 scene, the ISO/IEC 14496-1 initial object descriptor serves as 
tte initial access point to all associated streams. The initial object descriptor shaU be conveyed in the IOD 
descriptor located in the descriptor loop immediately following the program info length field in the 
Program Map Table of the program to which the scene is associated. It contains ES Descriptors identifying 
the scene description and object descriptor streams mat form part of this program. It may also contain 
fcS_Descnptars identifying one or more associated IPMP or OCI streams. Identification of streams is done 
by means of ES IDs as specified in clause 8 of ISO/IEC 14496-1. 

2.1 1 .3.1 1 P-STD Model for 14496 content 

Figure 2-6 shows the STD model when ISO/IEC 14496 systems data are carried in a Program Stream. 
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Figure 2-6 - P-STD model for ISO/EEC 14496 Systems stream 



The following notation is used in Figure 2-6 and its description : 

Bn is the input buffer for FlexMux stream n or for SL-packetized stream n. 

FBop is the FlexMux buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DBnp is the decoder buffer for the elementary stream in FlexMux channel p of FlexMux stream n 
DB n is the decoder buffer for elementary stream el 

Dg^ is the decoder for elementary stream in FlexMux channel p of FlexMux stream n 
D n is the decoder for elementary stream n. 

AnpO) is the j* access unit in elementary stream in FlexMux channel p of FlexMux stream n. A^Q) is 

indexed in decoding order. 
A„(j) is the j* access unit in elementary stream n. An(j) is indexed in decoding order. 
T dnpG) i s me decoding time, measured in seconds, in the system target decoder of the jth access unit in 

elementary stream in FlexMux channel p of FlexMux stream el 
TdnQ) is the decoding time, measured in seconds, in the system target decoder of the jth access unit in 

elementary stream n. 

Cnp(k) is the k* composition unit in elementary stream in FlexMux channel p of FlexMux stream n. 

Cnp(k) results from decoding A^Xj). C^Qs) is indexed in composition order. 
Ca(k) is the k* composition unit in elementary stream el Cn(k) results from decoding A n (j). C n (k) is 

indexed in composition order. 
tCnp(k) is the composition time, measured in seconds, in the system target decoder of the k* composition 

unit in elementary stream in FlexMux channel p of FlexMux stream n. 
tc n (k) is the composition time, measured in seconds, in the system target decoder of the k* composition 

unit in elementary stream n. 
t(i) Indicates the time in seconds at which the i* byte of the Program Stream enters the system target 

decoder. 
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2.11.3.11.1 Processing of FlexMux streams 

At the input of the STD each byte in the payload of PES packets carrying a FlexMux stream n is transferred 
instantaneously to buffer IV The i-th byte enters B 0 at time t(i). PES packet header bytes do not enter 
buffer B„ and may be used to control the system. The size of B n is specified by the P-STD buffer size field 
in the header of the PES packet that carries stream n_ ~ 

The FlexMux stream packet bytes in buffer B a are all delivered to their associated FlexMux buffer at the 
rate specified by the field finxRate encoded in the FlexMux stream and in compliance with the FlexMux 
buffer model defined in clause ll. 2.9 of ISO/IEC 14496- 1. Only FlexMux packet payload data bytes in 
FlexMux channel p of FlexMux stream n enter buffer FB^. FlexMux packet header bytes in FlexMux 
channel p of FlexMux stream n are discarded and may be used to control the system. The rare specified by 
the finxRate field shall be applicable for all FlexMux packets in the stream up to the next encountered 
FlexMux Clock Reference channel packet Bytes from the FlexMux Clock Reference channel are 
instantaneously removed and discarded and may be used to lock the ISO/DEC 14496 object time base to the 
STC. When there is no PES packet payload data present in B m no data is removed from B„. All data that 
enters B„ leaves it All PES packet payload bytes of stream n enter the FlexMux demultiplexer 
instantaneously upon leaving B n . 

2. 11.3.11.2 Definition of FlexMux Buffer, FB^ 

For each channel p of a FlexMux stream n, the size of FlexMux buffer FB™, is defined using the 
FmxBufferSize descriptor if a Program Stream Map is present in the Program Stream. FlexMux packet 
payload bytes are transferred from buffer FB^ to decoder buffer DB^ in compliance with the FlexMux 
buffer model defined in clause 11.2.9 of ISO/IEC 14496-1. Only SL packet payload bytes in FlexMux 
channel p of FlexMux stream n enter buffer DB^ The SL packet header bytes in FlexMux channel p of 
FlexMux stream n are discarded and may be used to control the system 

2. 11.3.11.3 Processing of SL-packetized streams 

At the input of the STD each byte in the payload of PES packets carrying an SL-packetized stream n is 
transferred instantaneously to buffer B„. The i-th byte enters B. at time t(i). PES packet header bytes do not 
enter buffer B» and may be used to control the system. The size of B» is specified by the P-STD buffer size 
field in the header of the PES packet that carries stream n. The SL-packetized stream bytes in buffer B~ are 
delivered to the decoder buffer DBn at the rate specified by the field instantBitRate encoded in the SL- 
packetized stream and in compliance with the System Decoder Model defined in clause 7.4 of ISO/DEC 
14496-1. The rate specified by the instantBitRate field shall be applicable for all data bytes in the SL- 
packetized stream up to the next encountered instantBitRate field. When there is no PES packet payload 
PF ^ nt m no data is removed from B n . All data that enters B, leaves it All bytes of stream n enter 
buffer DB n instantaneously upon leaving B^ with the exception of the SL packet headers. Bytes from the 
SL packet headers do not enter DB n and may be used to control the system. The size of decoder buffer DB- 
is given by the bufferSizeDB of the DecoderConfigDescriptor defined in ISO/IEC 14496-1. 

2.11.3.11.4 Buffer management 

Program Streams shall be constructed so that B„ does not overflow. FB^ shall not overflow. DB™ and DB- 
shall neither underflow nor overflow. Underflow ofDB^ occurs when one or more bytes of an access unit 
are not present m DB^ at the decoding time associated with this access unit Underflow of DB„ occurs 
when one or more bytes of an access unit are not present in DB n at the decoding time associated with this 
access unit -- 
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2.1 1 .3.1 2 Carriage within a Program Stream 

2.11.3.12.1 Overview 

A Program Stream contains only one program. ISO/TEC 14496 data can be conveyed in addition to the 
already defined stream types for such a program. As a special case, it is also possible that a Program Stream 
carries only ISO/TEC 14496 data. If a Program Stream Map is present, ISO/EEC 14496 content carried in 
the Program Stream shall be referenced as follows. Carriage of ISO/EEC 14496-1 scenes and associated 
ISO/TEC 14496 streams in SL and FlexMux packets is indicated by the appropriate stream id and by an 
initial object descriptor, the use of this descriptor is specified in clause 2.113.12.2. For each carried 
ISO/EEC 14496 stream the SL descriptor and die FMC descriptor shall specify the ES_ID. When the 
assignment of ES_ED values changes, the Program Stream Map, if present, shall be updated and die 
program_stream_map_version shall be incremented by 1 modulo 32. Note that in a Program Stream the 
ISO/TEC 14496 content may also be referenced by private means. 

For an example of a content access procedure for ISO/IEC 14496 program components within a Program 
Stream, see Annex R- 

2.11.3. 12.2 Initial Object Descriptor 

In case of carriage of an ISO/EEC 14496-1 scene, the ISO/IEC 14496 initial object descriptor serves as the 
initial access point to all associated streams. If a Program Stream Map is present in the Program Stream, the 
initial object descriptor shall be conveyed in the IOD descriptor that is located in the descriptor loop 
immediately following the program stream info length field. It contains ESJDescriptors identifying the 
scene description and object descriptor streams of the scene that form part of this program. It may also 
contain ESJDescriptors identifying one or more associated IPMP or OCI streams. Identification of streams 
is done by means of ES_EDs as specified in clause 8 of ISO/EEC 14496-1. In a Program Stream, the initial 
object descriptor may also be conveyed by private means. 
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9) Add after Annex Q: 

Annex R Carriage of ISO/IEC 14496 scenes in ITU-T Rec H 222 0 I 

ISO/IEC 13818-1 

(This annex does not form an integral part of this Recommendation | International Standard) 

R.1 Content access procedure for ISO/IEC 14496 program components within a 
Program Stream 

The following provides a reference receiver acquisition procedure for accessing ISO/IEC 14496 program 
elements in an ITU-T Rec. H.222.0 | ISO/DEC 13818-1 Program Stream. Here, it is assumed that the 
Program Stream includes a Program Stream Map (conveyed in a PES packet having stream id equal to 
OxBC): - 

1 . Acquire the Program Stream Map 

2. Identify the IOD descriptor in the first descriptor loop 

3. Identify the ES_E>s of the object descriptor, scene description and other streams described within the 
initial object descriptor 

4. Acquire the SL descriptor and FMC descriptor in the second descriptor loop for 
elementary_stream_ids OxFA and OxFB, as applicable. 

5. Generate from these descriptors a stream map table between ES_IDs and associated 
elementary_stream_id plus FlexMux channel, if applicable.. 

6. Locate the object descriptor stream using its ESID and the stream map table. 

7. Locate other streams described in the initial object descriptor using their ES ID and the stream man 
table. ~~ r 

8. Continuously monitor the object descriptor stream and identify ESJGDs of additional streams. 

9. Locate the additional streams using their ES ID and the stream map table. 

Figure R-l gives an example of ISO/IEC 14496 content in a Program Stream, consisting of object 
^ eam - description stream (BIFS-Conmiand), BIFS-Anim stream and IPMP stream. All 

ISO/IEC 14496 streams are multiplexed in a single FlexMux stream. 
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Figure R-l - Example of ISO/IEC 14496 content in a Program Stream 
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ISO/DEC 13818-l:1996/Amd.7:199x(E) 



R.2 Content access procedure for ISO/IEC 14496 program components within a 
I ransport Stream 

The following provides a reference receiver acquisition procedure for accessing ISO/IEC 14496 program 
elements in an ITU-T Rec. HL222.0 | ISO/IEC 13818-1 Transport Stream: 

1 . Acquire the Program Map Table for the desired program 

2. Identify the IOD descr^tor in the first descriptor loop 

3. Identify ^the ESJQDs of the object descriptor, scene description and other streams described within the 
initial object descriptor 



4. 

5. 



of^letmary' ^ ^ dB " , * te ^ * SCC ° Dd de ^ tor «* 

6. Locate the object descriptor stream using its ESJD and die stream map table 

7. Locate other streams described in the initial object descriptor using their ESJD and the stream map 



8. 

9. 



table. 

Continuously monitor the object descriptor stream and identify ES IDs of additional streams 
Locate the additional streams using their ESJD and the stream map table. 



£ j^S? ZT P ° f A 1SOnEC 14496 Ingram elements in a Transport Stream, consisting of 
stream, scene description stream (BIFS-Command), BIFS-Anim and IPMP elementary 
^J^ITS °° Stream m conve y ed by ^eans of ISOJECJ4496 sections, while 
flJS^Tp^? ^ « ementary streams are conveyed in PES packets referenced by two distinct 
elementary JID values, without the use of the ISO/IEC 14496-1 FlexMux tool. 



ITU-T Rec H.222.0 (1996)/Am«L7(199x E) -Z9~ 
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ISO/IEC 13818-1:1996/ Amd.7:199x(E) 
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PID> 0x0100 
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ISOJEC_14436_S«ctio* 



TS Paefcvt : PID =0x0111 



ISOJEC_14496_S»ctloa 



OD Stream 



TS PaclMt : PID *bc0112 

T 



Objoct Owsr riptrar Stream 



ES^Doscriptor ( 
ESJD =0x0013 
artroamTypo =?SD streanf 
spocifielnfcp "BIFS-Anlnf 



OfajactD«Beripto< 
ES.Deserlpter ( 
ESJD =0x0014 

tmTypo -IPMP streanf 



> 



Initial Object Descriptor 



E3_D«Jcriptor { 

ESJD =0x0011 
straamTypa ^SD streanf 
spaclfielnfbs "BIFS-corrf 



ESJD =0X0012 

i ==*OD streanf 



| TS Packet : PID =0x0100 



TS 



1st descriptor loop ( 
— lODjStescnptoO 



ijtype *Oxl3 

ID =0x0111 



2nd descriptor loop { 
SV_descripto{ 
ESJD =0x0011 



=0x13 

PID *bc0112 





_typ* eOxl2 

JHD =0x0113 
2nd descriptor loop £ 
SL_d9SCr3 pto^ 
ESJD =0x0013 



stream Jyp* =Oxl2 
etomentary.PID 4x0114 

2nd descriptor loop ( 
31* d«scr1pta< 
ES.ID =0x0014 



> 



TS Packet : PID 4x0113 



mjd ±1111 1010 



BIFS-Anbn! 



Figure R-2 - Example of ISO/EEC 14496 content in a Transport Stream 
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1. Introduction 

1.1. Purpose and scope 

1.2. Terminology 

1.2.1. Definitions 

1.2.2. Abbreviations and acronyms 

B,FS Binary Format for Scene description 

Ep G Electronic Program Guide 

MPEG Moving Picture Expert Group 

OD Object Descriptor 

PCR Program Clock Reference 

p ES Packet Elementary Stream 

PSI Program System Information 

1.3. Reference documents 

1.3.1. Controlling documents 

1449^r,May^ yStemS '' ' nfbrrnati0n Techno '°9y - Coding of Audiovisual Objects, Systems - ISO/IEC 

1449^May?^8 V ' de0 '" ,nf0mation Techno ^9y ~ Coding of Audiovisual Objects, Visual - ISO/IEC 

144 9 3°Ma7w8 AUdi ° lnfQmation Tecn "°l°9y " Coding of Audiovisual Objects, Audio - ISO/IEC 

l^^tS^S^ST '' Mbme ° n Technofo 9y- Coding of Audiovisual Objects. Systems - ISO/IEC 

t~ l5 l "^E^ „ s ^ e/ns Amendment 7 : Information Technology - Coding of Audiovisual Obiects 
Transport ofMPEG-4 data overMPEG-2 Systems', N3050. January 2000 "umovtsual Objects, 

1.3.2. Controlled documents 

1.3.3. Applicable documents 

1.4. Responsibility and approvals 
1.4.1. Responsibility 

Tr^eprojertje^te responsible for this document, the plans and the application of the plans. 
The group leader and the team involved in the development process are responsible for approvals 
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1.5- Overview 

This document presents the architecture for the transport of MPEG-4 data in MPEG-2 Systems. 
The end-to-end architecture is presented as well as the server and the terminal platforms. 
Application scenarios and a proposed set-up demonstration are also described. 
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2. Transport of MPEG-4 over MPEG-2 End-to-End System 
Architecture Overview 



2.1. Transport of MPEG-4 in MPEG-2 Systems Standard Specification 

The standard specification is fully described in "MPEG-2 Systems Amendment 7 : Information 
Technology - Coding of Audiovisual Objects, Transport of MPEG-4 data over MPEG-2 Systems" N3050 
January 2000. We propose here an architecture for validating this specification using an end-to-end system. ' 

2.1.1. Transport of MPEG*4 Streams in MPEG-2 Systems 

For "simple" MPEG-4 Audio and Video streams (without Scene description - BIFS - nor Object 
Descriptors), me method that is used is the same as for classical MPEG-2 A/V Streams. It consists in a 
mapping of MPEG-4 data in PES packets (MPEG-2 Systems Packetized Elementary Streams). A reference 
in the Program System Information (PSI table) is also needed to handle MPEG-4 stream type. 

The use of MPEG-4 Systems Synchronization Layer (SL) and/or FlexMux (FM) is not mandatory. 

2.1 .2. Transport of MPEG-4 Scenes in MPEG-2 Systems 

For more, complex applications with MPEG-4 Scenes involving MPEG-4 A/V streams and BIFS and 
Object Descnptors, the streaming data (MPEG-4 A/V) is still mapped to PES packets but the use of SL/FM 
is mandatory for the systems encapsulation before the PES layer. 

Moreover me BIFS and OD data are conveyed via MPEG-2 Sections (Tables that are re-transmitted 
regularly) using SL/FM format. 



2.2. Global End-to-End System Architecture 



MPEG-2 TS 



MUX/ 
TRANS- 
CODER 




MPEG-4 
Content 



SERVER 



MPEG-2 TS 



(carying MPEG-4) 



DEMUX 





Application 



TERMINAL 



Figure 1 : Global End-to-End System Architecture for the transport of 
MPEG-4 data over MPEG-2 Systems 
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The end-to-end system is composed of 2 main parts : the Server and the Terminal. A network layer 
(MPEG-2 Transport Stream) is the link between these 2 parts.* 

The role of the Server is to generate an MPEG-2 Transport Stream composed of MPEG-2 and MPEG-4 
A/V streams. The way of encapsulating MPEG-4 data in MEEG-2 Systems has to be compliant with the 
MPEG specification [5]. 

The role of the Terminal is to retrieve the MPEG-2 and MPEG-4 AN data from an input MPEG-2 TS 
stream and to use these data in a client application. 



2.3. Server Architecture 



2.3.1 . Platform Architecture 

The Server Platform is described on Figure 2 below. It is based on a trans-coding platform. 

The main input of this platform are : 

An existing MPEG-2 transport stream, 
- MPEG-4 data (audio, video, BIFS, OD, ...). 

The main output of the platform is a MPEG2- transport stream with MPEG-4 encapsulated in it. 



TS in 




MPEG-4 



Figure 2 : Platform Architecture 



The main blocks of such a system are described on Figure 3. These are : 

A bandwidth creation block that is in charge of creating bandwidth in the incoming transport 
stream in order to insert the MPEG-4 data, 

A MPEG-4 content creation block, that provide MPEG-4 content to be encapsulated, 
A MPEG-4 content encapsulation block that is in charge of the encapsulation of the MPEG-4 
data, provided by the MPEG-4 content creation block, in the MPEG-2 transport stream, provided 
by the bandwidth creation block. 
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DESC 



Control 



TSin 



Bandwidth 
creation 



MPEG-4 over IP 



MPEG-4 
content 
creation 



MPEG-4 
content 
encapsulation 



F/guns 3: Defa/Ys of the Trano Godiag Platform 



TSout 



de te Ne?rnTh^o^^ ayS t0 ' UnCti0na,ity ° f ** p,Wto » PreSented b, °<* s - S °™ 



are 



2.3.2. Bandwidth Creation Block 

.-Jisas&rsEj^ bandwid,h ta *• incomin9 lh - are nu " 

2.3.2.1. Null Packet Insertion 

<*«« m ?■ meth °^ 5 C 7 al f bandwidth consists in the insertion of null transport packet in the incoming 
iT^ finj * ° ut P ut toW* "it rate is higher than the incoming transport bit rate ^ 

nThSl^,2^^S Sclera 0 " * ^ * °" 4 " h - ^ « 



TSin 
bitrateR 




time 



Figure 4 : Bandwidth Creation by the Insertion of Null TS Packet 



LnLo^ 0 ^ 1 ! USed ' - dUe tQ packet contract,on ' must be taken to correctly re-stamped PCR fields in 
transport packets carrying the PCR. This effect is shown on Figure 5. »«mpeo r^K neias in 



PCR position A' 
in packet | 



PCR 

correction 



Figure 5 : PCR Correction 
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2.3.2.2. Bit-rate Trans-coding 

When the output transport bit rate cannot be increased, the previous method cannot be used. In that 
case it is possible to provide bandwidth for the video data by the trans-coding one of the MPEG-2 video 
data. 

Such a trans-coder is described on figure 6. In this system one of the program contains in the 
incoming transport stream is demux. Then its video is trans-coded in order to reduce its bit rate. Finally the 
video is remux in the incoming stream before going to the content encapsulation block. 



TSin 



Demux 



MPEG-2 video to 
trans-code : bitrate = R1 



unmodified 
data 




TSout 



MPEG-2 Video trans-coder 



* toMPE04 

content 
encapsulation 
block 



MPEG-2 video 
trans-coded: bit rate = 
R2 = R1 -delta 



Figure 6; Use of MPEG-2 Video Trans-Coding for Bandwidth Creation 



It should be noted that the mux part of the trans-coder might be a part of the MPEG-4 encapsulation 

block. 



2.3.3. MPEG-4 Content Creation Block 

Different way (Figure 3) may be used to provide MPEG-4 content (related to the MPEG-2 content) : 

tin-correlated content then MPEG-4 data are locally stored on the server, 

Remotely-correlated content the MPEG-4 may be transmitted (from a second remote server) over 
an IP link to the main server, 

Strongly correlated content where MPEG-4 data are extracted from the MPEG-2 content present 
in the incoming transport stream. In that case an MPEG-2 to MPEG-4 trans-coder is required. 



2.3.4. The MPEG-4 Content Encapsulation 

Two ways of encapsulation may be considered as specified in [5]. These are described in the following 
sections. 

2.3.4.1. Simple Encapsulation 

In the case where there is no correlation between MPEG-4 contents (no BIFS is required) then a 
simple encapsulation is required. It consists in the update of the MPEG-2 PSl table to indicate the presence 
of MPEG-4 data in the transport stream and of the encapsulation in PES packets of the Video and Audio 
content. 

This encapsulation is described on figure 7 below. 
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Figure 7: Simple MPEG-4 Data Encapsulation (Streams) 



2.3.4.2. Complex Encapsulation 

™, the case where MPEG-4 content is highly correlated (BIFS required) then a more complex 
SwrejIlS 1 18 neCSSSary - s y nchroni2ati on Layer and/or the Flex Mux layer of the MPEG-4 standard are 

As for the simple encapsulation method the MPEG-2 PSI table have to be updated and 
supplementary table have to be created to encapsulate BIFS information provided by the Flex Mux and/or 
oyncnronization Layer. 

h-in„ o A ^ d i 0 /- n< L Vid ^^ ntent has to 90 throu9h the nex Mux and/or Synchronization Layer before 
bemg encapsulated in the MPEG-2 PES packets. Such an encapsulation is described on Figure 8 below. 
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figure 8 : Complex MPEG-4 Data Encapsulation (Scenes) 
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It should be noted that for both encapsulations data synchronization is required. In the simple 
implementation to guarantee that the MPEG-4 data are correctly inserted in the stream and wont lead to a 
buffer overflow or underflow. This is also the case in the complex implementation. Moreover synchronization 
may be required at the BIFS level if it is correlated with the incoming MPEG-2 content. 

The clock recovery module described in the previous figures is part of the synchronization process. 



2.3.4.3. Server Architecture Summary. 

The previous sections have described different implementation of the different block of a MPEG-4 server 
platform. The different combinations lead to 12 different implementations (Table 1). 





ySfmpfe- , ,;7 ; \ '" - j- : 


Complex. O ' 


Bandwidth Creation 


Null Packets Encapsulation 


MPEG-2 Bit-rate Trans-coding 


MPEG-4 Content Creation 


Pre-encoded Content Stored 


MPEG-2 to MPEG^4 Content 




on Disk 


Trans-coding 


Encapsulation Method 


MPEG-4 A/V Streams 


MPEG-4 Scenes with SUFM 



7ab/e 1 : Server Architecture Summary 



2.4. Terminal Architecture 

2.4.1. Terminal Hardware Description 

The Terminal is based on PRL Set Top Box (STB) including a TriMedia processor chip and a DIVA MPEG-2 
Decoder chip (Rooster base). A view of PRL STB version TM1300 is presented in Figure 4. 




Figure 4 : PRL Set-Top Box 
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The STB Hardware specification comprises : 



4Z 



• CPU TM1300 (devices rated at 143MHz) 

• 32MB main memory 

• Acer M1 543 with UltraDMA and USB interfaces 

• Battery-backed Realtime clock 

• VGA output 

• DIVA MPEG-2 A/V decoder 

• DVB-S satellite tuner/demodulator integrated on main-board (MPEG2-TS from the tuner front- 
end can be chained through 2 PCMCIA sockets, supporting Common Interface (cT) 

• PSTN front-end on daughter-board 

" Sng DMA isochronous >W« *a TM Video-ln port - isochronous output via PCI 

Figure 5 present the STB main functional units. 




Figure 5 : PRL Set-Top box Functional Units 
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2.4.2. Terminal Functional Description 




Figure 6 : Terminal Functional Diagram 



All functions, but the MPEG-2 decoding, are performed in software on the TriMedia. 

The de-multiplexer extracts the data from the TS stream, performs the PES des-encapsulation and 
separates MPEG-2 data from MPEG-4 data. MPEG-2 A/V streams are decoded by the MPEG-2 decoder 
and the resulting frames are given to the application for rendering. 

The MPEG-2 TS demultiplexer has been modified to be able to identify and handle MPEG-4 data. The 
resulting MPEG-4 data are given to the MPEG-4 w EngineP for further processing (decoding). 



2.4.3. The MPEG-4 "Engine" at the Terminal Side 

The MPEG-4 engine, at the receiver side, performs the MPEG-4 AA/ decoding (Figure 7) and 
optionally the FM/SL des-encapsulation and the BIFS parser/decoder and the Object Descriptor interpreter, 
in case of the use of MPEG-4 Systems (for complex scenes, Figure 8). 




Figure 7 : MPEG-4 "Engine" for Simpie MPEG-4 AN Streams 
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Figure 8 : MPEG-4 'Engine' for more Complex MPEG-4 Scenes 



2.4.4. Applications 

fbrthe^d-^er 0 ' *" aPP ' iCati ° n " 10 ^ MPEG " 2 and ***** *» streams to buil <* ■ meaning*,, output 

The foreseen applications concern enhanced digital television combinino MPPn o aa/ et»» m * 
background program and enhanced information provid^ by MPE(S ^dips 3 

Concrete examples are listed here and shown on Figure 9 and 10 : 

• Interactive Electronic Program Guide (EPG), 

• "Immersive Broadcast", 

• 7-chanels Formula one. 
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F/gore 9 ; "immersive Broadcast" Application Concept 
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Figure 10 : MPEG-4 Interactive EPG 



2.5. Proposed Set-up Demonstrator 

A proposed set-up demonstrator is presented din Figure 1 1 . 




Figure 11 : Proposed Set-up Demonstrator 
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CLAIMS : 



1. 



A global end-to-end system architecture for the transport of MPEG-4 data 
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over MPEG-2 Systems, comprising the following parts : 

(A) a server, for generating an MPEG-2 Transport Stream composed of 
MPEG-2 audio/video streams ; 

(B) a terminal, for retrieving the MPEG-2 and MPEG-4 audio/video data from 
its input MPEG-2 Transport Stream, in view of a use of said data in a client application ; 

(C) a link between these two parts. 

2. In an architecture according to claim 1, a server platform receiving on the 
one hand an existing MPEG-2 transport stream and on the other hand MPEG-4 data and 
comprising the following main blocks : 

(1) a bandwidth creation block, for receiving said incoming MPEG-2 transport 
stream and creating bandwidth in it in order to insert said MPEG-4 data ; 

(2) an MPEG-4 content creation block, for receiving said MPEG-4 data and 
delivering the MPEG-4 content to be encapsulated ; 

(3) an MPEG-4 content encapsulation block, for encapsulating said MPEG-4 

content ; 

the output of said MPEG-4 content encapsulation block being the output signal of said 
server platform, in the form of an MPEG-2 transport stream with MPEG-4 content 
encapsulated in it ; 

(4) a fourth block for the control of the operations carried out by the three 

first ones. 

3. In a server platform according to claim 2, an encapsulation device consisting 
of updating means of the MPEG-2 PSI table, in order to indicate the presence of MPEG-4 
data in the transport stream and of the encapsulation in PES packets of the video and 
audio content, and data synchronization means, in order to guarantee that the MPEG-4 
data are correctly inserted in the stream, without risk of buffer overflow or underflow. 

4. In a server platform according to claim 3, an encapsulation device in which 
the Synchronization layer and the FlexMux layer of the MPEG-4 standard are required and 
an additional table has to be created n order to encapsulate the BIFS information 
provided by said FlexMux and Synchronization layers, the audio and video content having 
then to go through these FlexMux and Synchronization layers before being encapsulated 
in the MPEG-2 PES packets, and said data synchronization means being also required at 
the BIFS level. 
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